Goto

Collaborating Authors

 big impact


Small Models, Big Impact: Efficient Corpus and Graph-Based Adaptation of Small Multilingual Language Models for Low-Resource Languages

Gurgurov, Daniil, Vykopal, Ivan, van Genabith, Josef, Ostermann, Simon

arXiv.org Artificial Intelligence

Low-resource languages (LRLs) face significant challenges in natural language processing (NLP) due to limited data. While current state-of-the-art large language models (LLMs) still struggle with LRLs, smaller multilingual models (mLMs) such as mBERT and XLM-R offer greater promise due to a better fit of their capacity to low training data sizes. This study systematically investigates parameter-efficient adapter-based methods for adapting mLMs to LRLs, evaluating three architectures: Sequential Bottleneck, Invertible Bottleneck, and Low-Rank Adaptation. Using unstructured text from GlotCC and structured knowledge from ConceptNet, we show that small adaptation datasets (e.g., up to 1 GB of free-text or a few MB of knowledge graph data) yield gains in intrinsic (masked language modeling) and extrinsic tasks (topic classification, sentiment analysis, and named entity recognition). We find that Sequential Bottleneck adapters excel in language modeling, while Invertible Bottleneck adapters slightly outperform other methods on downstream tasks due to better embedding alignment and larger parameter counts. Adapter-based methods match or outperform full fine-tuning while using far fewer parameters, and smaller mLMs prove more effective for LRLs than massive LLMs like LLaMA-3, GPT-4, and DeepSeek-R1-based distilled models. While adaptation improves performance, pre-training data size remains the dominant factor, especially for languages with extensive pre-training coverage.


Crumple.News : The Oregon Trail: Simple Game with a Big Impact

#artificialintelligence

In 1971, three student teachers at Carleton College in Minnesota created a computer game to teach their students about the westward expansion. Don Rawitsch, Bill Heinemann, and Paul Dillenberger programmed the game in BASIC language on an HP 2100 minicomputer with only 32 kilobytes of memory. The game was designed to simulate the experience of a family traveling from Missouri to Oregon in 1848 and teach students about the challenges faced by pioneers on the Oregon Trail. The game became popular in classrooms across the United States and eventually was published by MECC (Minnesota Educational Computing Consortium) in 1985. Over the years, "The Oregon Trail" has undergone numerous updates and re-releases for various platforms. The original version of the game was text-based, and players had to use the arrow keys to navigate their wagon.


Survey says AI and machine learning tools like ChatGPT will shake up finance sector - Business Leader News

#artificialintelligence

Artificial intelligence and machine learning tools like ChatGPT are set to shake up the finance sector, according to a new poll from JP Morgan. Over half of traders surveyed in JP Morgan's eTrading survey felt that AI and machine learning will be the most influential technology over the next three years, a rise of 25 percent from last year. This was a major change from last year when mobile trading applications topped the survey with 29 percent and blockchain technology scored 25 percent. AI and machine learning has had a big impact on traders in recent years. The technology can analyse and process huge amounts of data far more accurately than humans, identifying patterns and trends.


Think globally, act locally: Starting small with AI can make a big impact

#artificialintelligence

Check out all the on-demand sessions from the Intelligent Security Summit here. AI has made and will continue to make significant headlines. Most of these are fairly sensational; AI is becoming sentient; AI-generated art wins a contest; AI can now compose music (and more). However, what rarely makes the headlines is just how transformational AI can be when it comes to business --specifically, how AI can help brands connect with their customers without becoming a flashy sci-fi headline. Because of the general "sci-fi" perception of AI, many business leaders haven't seriously considered how to apply it to their business beyond data analytics or cutting-edge research labs.

  Industry:

Council Post: Small Data, Big Impact: Making The Most Of AI With Less

#artificialintelligence

Headlines have reiterated that investments in big data have continued to increase, and the big data story they're selling us on just keeps getting bigger. For the last decade, the big data story has been plastered across headlines and weaved into organizations' IT models as the end-all-be-all solution. We've been sold on the idea that AI and big data will together be what drives modern businesses down the path to success, helping these companies thrive in today's digital-first, consumer-driven environment. Across almost every industry, from financial services to healthcare to real estate and beyond, this story told us that all problems must be solved by more computing power and more data analysis -- AI and big data. And that's a good thing since companies have been, for years, aggregating massive data sets to fuel algorithms to create positive experiences and outcomes.


AI is having a big impact, but not how you think

#artificialintelligence

Artificial intelligence has grabbed headlines for the past few years, but too often the press oversells the risks and rewards of AI. We read about AI's inevitable bias, and its deadly use in war. Of course, we also read the positive, like a Google computer beating the world's best Go players. But these stories fail to accurately reflect the best uses of AI today. I wrote years ago that IBM needed to stop pitching its Watson as a miracle cure for most everything, and instead position it for more pedestrian tasks. In like manner, we'd do better to celebrate AI adoption in small steps that add up to major savings--like food and waste and other sectors.


How AI Will Impact The Future Of Work And Life

#artificialintelligence

AI, or artificial intelligence, seems to be on the tip of everyone's tongue these days. While I've been aware of this major trend in tech development for a while, I've noticed AI appearing more and more as one of the most in-demand areas of expertise for job seekers. I'm sure that for many of us, the term "AI" conjures up sci-fi fantasies or fear about robots taking over the world. The depictions of AI in the media have run the gamut, and while no one can predict exactly how it will evolve in the future, the current trends and developments paint a much different picture of how AI will become part of our lives. In reality, AI is already at work all around us, impacting everything from our search results, to our online dating prospects, to the way we shop.


How AI Will Impact The Future Of Work And Life

#artificialintelligence

AI, or artificial intelligence, seems to be on the tip of everyone's tongue these days. While I've been aware of this major trend in tech development for a while, I've noticed AI appearing more and more as one of the most in-demand areas of expertise for job seekers. I'm sure that for many of us, the term "AI" conjures up sci-fi fantasies or fear about robots taking over the world. The depictions of AI in the media have run the gamut, and while no one can predict exactly how it will evolve in the future, the current trends and developments paint a much different picture of how AI will become part of our lives. In reality, AI is already at work all around us, impacting everything from our search results, to our online dating prospects, to the way we shop.


Artificial intelligence expected to have a big impact on white collar jobs 7wData

#artificialintelligence

A new report by the Brookings Institution counters previous analyses showing less-educated, lower-wage workers would be most exposed to automation. Better educated, better paid white collar workers will be the most affected by artificial intelligence (AI), according to a newly releasedreport by the Brookings Institution. The report goes against previous findings of Brookings' and other research that shows less educated and lower-wage workers will be most impacted by robots. Stanford University researcher Michael Webb's approach was to take the text of patents to identify the capabilities of AI, and then quantify the extent to which each occupation involves these technologies. Webb used natural language processing to quantify the overlap between patent texts and job description text and came up with an exposure score for each job.


Artificial intelligence expected to have a big impact on white collar jobs

#artificialintelligence

Better educated, better paid white collar workers will be the most affected by artificial intelligence (AI), according to a newly released report by the Brookings Institution. The report goes against previous findings of Brookings' and other research that shows less educated and lower-wage workers will be most impacted by robots. Stanford University researcher Michael Webb's approach was to take the text of patents to identify the capabilities of AI, and then quantify the extent to which each occupation involves these technologies. Webb used natural language processing to quantify the overlap between patent texts and job description text and came up with an exposure score for each job. Out of the 769 occupational descriptions Webb analyzed, 740 "contain a capability pair match with AI patent language, meaning at least one or more of its tasks could potentially be exposed to, complemented by, or completed by AI,'' the report noted. "Importantly, this does not mean such tasks will be ...